Claims 



1. A method for determining when a user has ceased inputting data, the method 
comprising the steps of: 

5 receiving an input from a user; 

accessing a plurality of templates from a database; 

determining if all inputs received from the user fill any templates from the 
database; and 

determining that the user has ceased inputting data if the user's inputs fill 
10 any template from the database. 

2. The method of claim 1 further comprising the steps of: 

determining if predetermined amount of time has passed; and 
determining that the user has ceased inputting data if the predetermined 
1 5 amount of time has passed. 

3. The method of claim 1 wherein the step of receiving the input from the user 
comprises the step of receiving a multi-modal input from the user. 

20 4. The method of claim 3 wherein the step of receiving the multi-modal input 
from the user comprises the step of receiving a multimodal input from the group 
consisting of a text input, a speech input, and a handwritten input. 

5. The method of claim 1 wherein the step of accessing the plurality of templates 
25 comprises the step of accessing a plurality of semantic templates. 

6. The method of claim 1 wherein the step of accessing the plurality of templates 
comprises the step of accessing a plurality of templates comprising combinations 
of possible user inputs and their possible mode of input. 

30 

7. The method of claim 1 further comprising the step of dynamically updating 
templates from the database. 

8. The method of claim 7 wherein the step of dynamically updating templates 
35 from the database comprises the step of dynamically updating templates based on 

8 



a characteristic taken from the group consisting of available modes of input, an 
expected response from the user, a list of discourse obligations that constrain what 
the user can input in the next dialog turn, and the history and the current status of 
the task(s) that the user is working on. 

5 

9. A method comprising the steps of: 

receiving a plurality of user inputs; 

determining a content of the input for each of the user inputs; 

determining a mode of input for each of the user inputs; 
10 accessing a plurality of templates; 

determining if the content and mode of the user inputs fill a template from 
the plurality of templates; and 

determining that the user has ceased inputting data if the user's inputs fill 
any template. 

15 

10. The method of claim 9 further comprising the steps of: 

determining if predetermined amount of time has passed; and 
determining that the user has ceased inputting data if the predetermined 
amount of time has passed. 

20 

11. The method of claim 9 wherein the step of receiving the plurality of user 
inputs comprises the step of receiving a plurality of multi-modal inputs from the 
user. 

25 12. The method of claim 11 wherein the step of receiving the plurality of user 
inputs comprises the step of receiving a plurality of multimodal inputs from the 
group consisting of a text input, a speech input, and a handwritten input. 

13. The method of claim 9 wherein the step of accessing the plurality of templates 
30 comprises the step of accessing a plurality of semantic templates. 

14. The method of claim 9 wherein the step of accessing the plurality of templates 
comprises the step of accessing a plurality of templates comprising combinations 
of possible user inputs and their possible mode of input. 

35 
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15. The method of claim 9 further comprising the step of dynamically updating 
the plurality of templates. 

16. The method of claim 15 wherein the step of dynamically updating the plurality 
of templates comprises the step of dynamically updating templates based on a 
characteristic taken from the group consisting of available modes of input, an 
expected response from the user, a list of discourse obligations that constrain what 
the user can input in the next dialog turn, and the history and the current status of 
the task(s) that the user is working on. 

17. An apparatus comprising: 

a user interface having a plurality of multi-modal user inputs; 
a template database outputting templates; and 

a multi-modal input fusion (MMIF) module receiving the multi-modal user 
inputs and the templates, and determining if a content and mode of inputs fills a 
template received from the database. 

18. The apparatus of claim 17 wherein: 

the MMIF module determines that a user has ceased inputting data when 
the content and mode of inputs fill a template received from the database, or a 
predetermined amount of time has passed since receiving a last input from the 
user. 

19. The apparatus of claim 17 wherein the templates comprise semantic templates. 

20. The apparatus of claim 17 further comprising a template generator 
dynamically updating the templates. 
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